Repository navigation
Trace genai dashboard - #398
Merged
Merged
Conversation
Dependency ReviewThe following issues were found:
License IssuesCargo.lock
Scanned Manifest FilesCargo.lock
|
Fixes format refactor tracing
thorrester
force-pushed
the
trace-genai-dashboard
branch
from
May 3, 2026 23:28
d170a66 to
63c522e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Short Summary
Adds a GenAI tab to the trace detail view — a dedicated panel showing LLM-specific metrics (token usage, cost, operation breakdown, model usage, eval results, tool calls) pulled from Scouter's GenAI span data. Also expands the trace filter sidebar to support namespace, version, and instance_id facets alongside the existing service/status filters.
Context
GenAI trace panel (
TraceDetailContent,SpanGenAiPanel,TraceGenAiPanel)The trace waterfall had no way to surface LLM-specific span attributes. When a trace contains GenAI spans (
has_genai_spansfrom the Scouter response), a "GenAI" tab now appears next to the existing Waterfall and Map tabs. Selecting it rendersTraceGenAiPanelwith aggregate metrics across the trace, andSpanGenAiPanelin the detail pane for per-span breakdowns (messages, eval results, tool call blocks, params, raw attributes).TraceDetailContentreceivesgenai: GenAiTraceMetricsResponse | nullandgenAiBySpanId: Record<string, GenAiSpanRecord>— when null or whenhas_genai_spansis false, the tab is hidden entirely.Facet sidebar expansion (
FacetSidebar,TraceDashboard)Before: facet sidebar offered Service and Status filters only.
After: Service, Namespace, Version, Instance ID, and Status.
TraceDashboardnow fires four parallel facet requests instead of two (service_namespace,service_version,service_instance_idfilters each resolved independently).FacetSidebaraccepts the new facet arrays and their set/clear callbacks. When Scouter returns no facet values for a dimension (e.g. namespace isn't indexed yet), the sidebar falls back to a free-text input with an Apply button.Agent observability charts (
charts.ts)Added
buildOperationBarChart— a horizontal bar Chart.js config ranked by span count — consumed by the newTraceGenAiPaneloperation breakdown card.Rust genai route (
scouter/genai/route.rs)GenAiDashboardRequestwas defined locally and is now imported fromopsml-types.is_valid_trace_idpreviously rejected UUID-format trace IDs (containing hyphens) because it checked raw chars against hex; fixed by stripping hyphens before validation. Unit test added for the boundary cases including UUID format.Python tracing (
opsml/scouter/tracing/__init__.py)Replaced
set_entity+add_queue_itemonScouterSpanwithattach_eval— a structured method that attaches an eval record to the trace viaprofile_uidand a dict/Pydantic context payload. Baggage propagation refactored from_current_context(which only fetched the ambient OTel context) to_apply_baggage_to_context(which builds a new context with all provided baggage items applied). This fixes cases where baggage was silently dropped when a non-None context was passed.init_traceris replaced byconfigure_tracing+get_tracer/reset_tracer_providerto align with the updated Scouter client API.ScouterResourceConfigandTraceFiltersadded to exports.AppState/ service loading (opsml_app/src/app.rs)Added
normalize_drift_pathsto resolve relative drift profile paths before passing them toScouterQueue::from_path_rs. Previously, drift paths stored incard_map.jsonas relative paths (e.g../profiles/drift.json) would fail at runtime when the working directory differed from the service artifact directory.from_specnow checksapp_state().is_offline()and skips registration when offline (instead of always registering).crates/opsml_server/opsml_ui/src/lib/components/trace/genai/SpanGenAiPanel.sveltecrates/opsml_server/opsml_ui/src/lib/components/trace/genai/TraceGenAiPanel.sveltecrates/opsml_server/opsml_ui/src/lib/components/trace/TraceDetailContent.sveltegenaiandgenAiBySpanIdpropscrates/opsml_server/opsml_ui/src/lib/components/trace/TraceDashboard.sveltecrates/opsml_server/opsml_ui/src/lib/components/trace/filters/FacetSidebar.sveltecrates/opsml_server/opsml_ui/src/lib/components/card/agent/observability/charts.tsbuildOperationBarChartcrates/opsml_server/src/core/scouter/genai/route.rsGenAiDashboardRequestto types; fixes UUID trace ID validationcrates/opsml_server/tests/api/scouter/trace.rspy-opsml/python/opsml/scouter/tracing/__init__.pyset_entity/add_queue_itemwithattach_eval; fixes baggage propagationcrates/opsml_app/src/app.rsnormalize_drift_paths; offline-awarefrom_speccrates/opsml_app/src/types.rsScouterResourceConfig,ServiceInfo, and instance ID resolutionIs this a Breaking Change?
Yes —
ScouterSpan.set_entityandScouterSpan.add_queue_itemare removed from the Python tracing API. Callers using those methods must migrate toattach_eval. Theinit_tracerfunction is also removed; callers must switch toconfigure_tracing+get_tracer. All other Rust HTTP contracts, SvelteKit routes, and server handler signatures are unchanged.